Skip to content

🔍 .NET Aspire Demo with Qdrant Search & GitHub Models Integration #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Jun 26, 2025
Merged

🔍 .NET Aspire Demo with Qdrant Search & GitHub Models Integration #28

merged 6 commits into from
Jun 26, 2025

Conversation

@jongalloway
Copy link
Collaborator

@jongalloway jongalloway commented Jun 26, 2025 •
edited
Loading

Add Qdrant Vector Search & GitHub Models to AspireDemo

What This Does

Adds real semantic search capabilities to the .NET Aspire demo using:

  • Qdrant vector database for storing and searching document embeddings
  • GitHub Models integration for generating embeddings (with simple fallback)
  • RSS feed ingestion from .NET blogs to populate search content
  • Web UI for searching, configuring tokens, and ingesting content

Key Features

Search Experience

  • Search through ingested .NET blog content using natural language
  • Real-time similarity search with configurable thresholds
  • Automatic embedding type detection and threshold adjustment

GitHub Models Integration

  • Optional GitHub token input for accessing GitHub Models embeddings
  • Automatic fallback to simple hash-based embeddings when token unavailable
  • Session storage for token persistence

Content Management

  • RSS feed ingestion from .NET developer blogs
  • Optimized processing (25 most recent items, 5-minute timeout)
  • Token-aware embedding generation during ingestion

Developer Tools

  • Search diagnostics and embedding comparison tools
  • Document browser for viewing ingested content
  • Performance monitoring with OpenTelemetry integration

Usage

  1. Start the application: dotnet run (Qdrant container starts automatically via Aspire)
  2. Ingest content: Use "RSS Ingestion" page to load .NET blog content
  3. Search: Use "Vector Search" page to search through content
  4. Optional: Add GitHub token for better embedding quality

Known Issues

  • GitHub Models embeddings currently show poor semantic similarity compared to simple embeddings
  • This suggests potential API integration issues that need investigation

Technical Details

  • Frontend: Blazor with session-based token management
  • Backend: .NET Aspire with Qdrant integration
  • Architecture: Dynamic embedding service selection based on token availability
  • Monitoring: OpenTelemetry tracing with correlation IDs

This transforms the AspireDemo from a basic proof-of-concept into a functional vector search application while maintaining all existing functionality.

Addresses Issues: #1 (Backend Data Sources), #2 (AI/LLM Integration)
Related to: #4 (Authentication patterns)
Files changed: 60 files, 5,187 additions
Dependencies: Requires Qdrant container (handled by Aspire)
Breaking changes: None

@jongalloway
Add initial implementation of NLWebNet Frontend with vector search fu…
5e06591 
…nctionality

- Created Test.razor for a simple test page with time update and alert features.
- Added TestInteractive.razor for an interactive counter page.
- Implemented VectorSearch.razor for semantic search with UI for query input and results display.
- Introduced Routes.razor for routing configuration.
- Added _Imports.razor for common using directives across components.
- Defined global usings in GlobalUsings.cs for cleaner code.
- Created ApiModels.cs to define data models for API interactions.
- Set up NLWebNet.Frontend.csproj with necessary project properties and references.
- Implemented Program.cs for application startup and service configuration.
- Configured launchSettings.json for local development.
- Added appsettings.json for application configuration.
- Created app.css for styling the frontend application.
- Developed Extensions.cs for common service defaults including health checks and OpenTelemetry.
- Set up ServiceDefaults.csproj with required packages for service defaults.
- Added start-aspire.bat for easy application startup.
- Created interfaces and service classes for vector storage.
- Removed obsolete appsettings.Development.json.
- Added AskController and McpController for handling API requests.
- Created unit test classes for AskController and McpController.
@jongalloway
feat: Enhance application layout and navigation
f4c81ff 
- Added a sidebar with a navigation menu in MainLayout.razor.
- Updated NavMenu.razor to change the "Test" link to "Configuration".
- Introduced a new Configuration.razor page for managing GitHub token settings and providing information about semantic search.
- Modified RssIngestion.razor to improve error handling and user feedback with success/error messages.
- Removed obsolete Test and TestInteractive pages.
- Updated VectorSearch.razor to utilize new API and embedding configuration services.
- Created ApiService and EmbeddingConfigurationService for better API interaction and configuration management.
- Updated Program.cs to register new services and improve dependency injection.
- Added detailed setup instructions for vector search in VECTOR_SEARCH_SETUP.md.
- Improved CSS styles for better layout and responsiveness.
@jongalloway
feat: Improve logging and configuration handling in frontend and back…
c457849 
…end services
@jongalloway
Enhance RSS Ingestion and Vector Search Features
8075a6a 
- Updated RssIngestion.razor to improve demo feed ingestion with GitHub token support and enhanced logging.
- Modified VectorSearch.razor to include dynamic threshold adjustments based on GitHub token presence, added debug information, and improved user interaction with search functionality.
- Configured additional OpenTelemetry sources for better tracing in Program.cs.
- Enhanced ApiService with detailed logging and tracing for search operations, including correlation IDs for better tracking.
- Updated appsettings.json to adjust logging levels for System.Net.Http to Information for improved visibility.
@jongalloway
feat: Add diagnostic endpoints for embedding and search analysis, and…
6b8c356 
… implement document browsing functionality
@jongalloway jongalloway changed the title 🚀 .NET Aspire Demo with Qdrant Vector Search & GitHub Models Integration Add Qdrant Vector Search & GitHub Models to AspireDemo Jun 26, 2025
@jongalloway jongalloway changed the title Add Qdrant Vector Search & GitHub Models to AspireDemo 🔍 Add Qdrant Vector Search & GitHub Models to AspireDemo Jun 26, 2025
@jongalloway jongalloway requested a review from Copilot June 26, 2025 03:20
@jongalloway jongalloway changed the title 🔍 Add Qdrant Vector Search & GitHub Models to AspireDemo 🔍 .NET Aspire Demo with Qdrant Search & GitHub Models Integration Jun 26, 2025
Copilot AI reviewed Jun 26, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR transforms the AspireDemo sample into a full semantic vector search application using Qdrant and GitHub Models, including a Blazor UI, ingestion workflows, and service defaults.

  • Adds Qdrant vector database integration and vector storage service
  • Implements composite embedding service (GitHub Models → OpenAI → simple fallback)
  • Builds a Blazor front-end with pages for RSS ingestion, vector search, statistics, and configuration

Reviewed Changes

Copilot reviewed 47 out of 60 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
samples/Demo/appsettings.Development.json Removed legacy local dev settings
samples/AspireHost/Program.cs Cleared out default AspireHost sample bootstrap
samples/AspireDemo/start-aspire.bat Added a quick-start batch script
samples/AspireDemo/VECTOR_SEARCH_SETUP.md Documented vector search configuration options
samples/AspireDemo/ServiceDefaults/ServiceDefaults.csproj Added shared Aspire defaults project
samples/AspireDemo/ServiceDefaults/Extensions.cs Factored out default health checks, resilience, OpenTelemetry
samples/AspireDemo/NLWebNet.Frontend/wwwroot/css/app.css Added base styles and layout CSS
samples/AspireDemo/NLWebNet.Frontend/Program.cs Wired up service defaults, HttpClients, and DI
samples/AspireDemo/NLWebNet.Frontend/Services/EmbeddingConfigurationService.cs Session-backed GitHub token management
samples/AspireDemo/NLWebNet.Frontend/Services/ApiService.cs Front-end API client with tracing & logging
samples/AspireDemo/NLWebNet.Frontend/Components/Pages/VectorSearch.razor Main vector search UI with search & diagnostic tools
samples/AspireDemo/NLWebNet.AspireApp/Services/QdrantVectorStorageService.cs Qdrant collection initialization and CRUD
samples/AspireDemo/NLWebNet.AspireApp/Services/GitHubModelsEmbeddingService.cs Embedding service for GitHub Models API
samples/AspireDemo/NLWebNet.AspireApp/Program.cs .NET minimal APIs mapping ingestion, search, diagnostics

samples/AspireDemo/NLWebNet.AspireApp/Services/GitHubModelsEmbeddingService.cs
httpClient.BaseAddress = _httpClient.BaseAddress;
httpClient.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", githubToken);
httpClient.DefaultRequestHeaders.Add("User-Agent", "NLWebNet-AspireDemo");
}
Copy link

Copilot AI Jun 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating and disposing a new HttpClient instance manually bypasses IHttpClientFactory benefits and can lead to socket exhaustion. Consider injecting IHttpClientFactory and using CreateClient for per-request token configuration.

Copilot uses AI. Check for mistakes.
samples/AspireDemo/NLWebNet.Frontend/Program.cs
// Use the actual API URL from Aspire dashboard
client.BaseAddress = new Uri("https://localhost:7220");
client.Timeout = TimeSpan.FromMinutes(5); // 5 minutes for RSS ingestion
});
Copy link

Copilot AI Jun 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Registering the default HttpClient as the "DirectApiClient" may confuse consumers expecting the named "ApiClient". Consider injecting IHttpClientFactory or aligning the default with the primary ApiClient registration.

Copilot uses AI. Check for mistakes.
@jongalloway jongalloway merged commit a1e9cf9 into main Jun 26, 2025
8 checks passed
@jongalloway jongalloway deleted the add-qdrant branch July 1, 2025 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jongalloway